In many high-end digital cameras, a light beam is partitioned by a beam splitter into three component single color beams. Each component beam strikes a light sensor array such as a charge-coupled device (light sensor array) or Complementary Metal Oxide Semi-conductor (CMOS) sensor. These light sensors are typically comprised of hundreds of thousands of photo sensitive sensor elements. Each sensor element senses the intensity of the colored light striking it. The greater the intensity of the incoming light, the greater the resulting charge on that sensor element. These individual charges are converted into electrical signals which are further converted and encoded into digital data. The digital data is combined such that each pixel location of the resulting image includes a red, green and blue value.
Lower-end digital cameras use a single light sensor-array in conjunction with a color filter array (CFA). The image light passes through conventional optics and is directed onto a CFA, which can be a coating on the light sensor array or separate from the light sensor array. The CFA comprises an array of discrete color filter elements, each corresponding to a sensor element of the light sensor array. Each discrete color filter element filters all light except one color. Each filter element for a typical CFA normally allows one of three colors through. The most common CFA systems are red-green-blue ("RGB") and magenta-green-cyan-yellow ("Mg-G-Cy-Y"). Thus, a light sensor array sensor element aligned to receive light filtered by a red color filter element of the CFA, generates an intensity signal proportional to the red light in the polychromatic light incident on the filter element. Traditionally, a CFA includes more luminance filter elements (e.g., green in an RGB system) than chrominance filter elements (e.g. red or blue in an RGB system). Luminance is sampled at a greater frequency as the human eye is more sensitive to luminance than chrominance information.
The data collected by a single light sensor array system is sparsely sampled because each sensor element only senses one color wavelength and not all three. The data collected by a single light sensor array system can be thought of as three partially populated component color arrays, each array including all information received for a single color by the light sensor array. To generate a complete color image, a value for each location in each color component array is required.
Known interpolation methods (e.g., bilinear or cubic spline) exist that may be used to complete component color arrays for a color image. An improvement over the bilinear method is disclosed in U.S. Pat. No. 5,382,976 to Hibbard (the "'976 Patent"). That patent is directed to the interpolation of array elements for a high frequency color component array. As is well known, the green color components of an RGB color component system, which are sometimes referred to as the luminance components, are thought to be the color component to which the human eye is most sensitive. This sensitivity means that the human eye discerns most of the detail in an image from the green color component or luminance of an image. So that a light sensor array captures detail to which the human eye responds, most CFAs include at least twice as many green or luminance color filter elements as other color filter elements.
The interpolation technique of the '976 Patent generates green or luminance intensity values for those array elements in the green component array which did not receive a green intensity value. The technique of that patent uses gradients to select the existing luminance values in the vicinity of array elements for which a value is to be interpolated. The selected luminance values are then used to generate a luminance value for the missing array element. The completed luminance array and the existing chrominance values in the other color component arrays are used to interpolate array element intensity values for the color components other than green.
While the technique disclosed in U.S. Pat. No. 5,382,976 improves the image quality over those derived using the bilinear interpolation method alone, artifacts still result in images generated from this method. In particular, color artifacts arise when use of a gradient results in an interpolated value for a green array element which is lower than it should be to accurately represent the image and the corresponding red or blue component improperly dominates the composite color image. This may be particularly noticeable at edges where an array element should be included in a horizontal or vertical edge but the bilinear interpolations for the elements of the chroma color component array includes intensities in both the horizontal and vertical directions.
Other variations of the bilinear method are used to predict missing pixel intensity values. For example, U.S. Pat. No. 4,630,307 to Cok (the "'678 Patent") discloses a method of interpolation in which a median of four neighboring values is used if the value is located at an edge. Otherwise, the missing array element is interpolated using a median of those values and a clip function. The values used for the clip function depend upon the particular pattern to which the missing pixel location belongs. While this method improves pixel interpolation for some array elements, especially those located at some edges, the resulting images are still subject to color artifacts.
What is needed is a method and apparatus for predicting values to complete partially populated single color arrays without generating numerous color artifacts.